Content Distribution Infrastructures

Since the early days of the world wide web (WWW), the information infrastructure provided over the Internet has improved considerably. The simplicity of offering and accessing data on the WWW and the increase of commercial uses of the Internet are major reasons for this development. During this development, it was discovered that shifts in popularity of content and services offered over the web lead to unbalanced load. A sudden increase in popularity can occur (slashdot effect) and is very hard to predict. Therefore it is also very hard to scale these servers and their connection to the Internet at the right time. To overcome these problems, hierarchies of cache servers such as harvest and squid have been set up to alleviated general load, but the large amount of content has limited their effectiveness in many situations. For that reason, controlled pre-distribution of content on behalf of the content providers has been offered as commercial service. To offer the service, servers are installed in the networks of different Internet service providers around the world. The servers can cooperate but they are only connected through the Internet. An infrastructure for this kind of service is known as content distribution network (CDN). Offering content through such a CDN has two advantages. First, several providers’ content is offered from the same CDN and only the content of very few providers will experience a sudden increase in popularity at the same time. By offering the content through a CDN, not every content provider must maintain servers that can cope with the load that he experiences at times of very high popularity. Instead, the CDN hosting the content will experienced a multiplexed load, and it can be scaled to cope with high load for some but not all content providers. Second, content in the CDN may be replicated over several hosting centers and accesses to content can be redirected to the closest one, thus reducing network load and access latency. By now, these commercial content hosting CDNs that offer access to discrete media constitute the classical form kind of CDNs. There are, however, other forms of content distribution that are covered by the term CDN as well. One form is used to distribute a different kind of content, namely live data streams that are transmitted from a single source to a large audience. Another form distributes discrete media as well, but uses the peer-to-peer (P2P) model in which equal nodes collaborate for the distribution, and in which nodes are often owned by individuals. Even though commercial CDNs for live and stored content and P2P systems have been investigated separately so far, they share characteristics, for example that they consist of nodes that are connected through the Internet but typically not directly to each other. The integration of these approaches is therefore a challenge to researchers. Another is the increased importance of multimedia content and interactive applications, which will impose new problems on CDNs. This seminar has brought together researchers who address the challenges that lie in the improvement of CDNs. Among the challenges faced are those related to scaling of the CDN infrastructure, the use of appropriate techniques and tools, and the management of the growing CDNs. Several attendees of the seminar presented ideas and results concernings this topic. Topics covered in presentations of participants’ current research include system and network support for scalability of content and CDN nodes, the communication between nodes and from the nodes to end-systems. The topics included dimensioning, scaling, configuration and reconfiguration of distribution networks for both, hierarchical CDNs and those that follow the P2P model, and reports on the needs of variations applications that rely on CDNs now or in the future.